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ABSTRACT 



Teaching College Composition with Computers: 
A Program Evaluation Study 

This program evaluation was undertaken at Southern Illinois University-Carbondale (SIU-C) 
to assess the broad, measureable effects of using computers to teach introductory college 
composition. Twenty-four classes were studied**twelve control classes and twelve 
experimental— with the experimental computer classes meeting in the lab for half of their 
instructional time. Data on the success of the program were collected from a range of 
sources: pre- and posttests of student writing under both impromptu and take-home 
conditions; pre- and posttests of writing anxiety; records on attendance, tardiness, 
withdrawals, and homework and essay assignment completion; end-of-term course evaluation 
by both teachers and students; and self-report data collected from teacher meetings and 
teacher logs. Results favored the use of computers, with computer students revising and 
improving their posttest essays (especially discourse-level features) at levels that are 
significantly better than regular students. Those students in experimental sections who chose 
to compose on computers at the end of the term outperformed the group as a whole and 
performed significantly better than those experimental students who chose to compose with 
pen and paper. Attitudinal data from both students and teachers also favored the use of 
computers. 



Teaching College Composition with Computers: 
A Program Evaluation Study 

Nfany college English departments have begun using the microcomputer in 
composition classrooms with the hope that it will improve student writing (especially 
revision processes), encourage better attitudes toward writing, and perhaps stimulate 
a collaborative learning environment. We have a wealth of ^If-report data which 
suggests that the introduction of computers does lead to a range of benefits for 
student writers (Arkin & Gallagher, 1984; Feldman, 1984; Hunter, 1983; Moore, 
1985; Nash & Schwartz, 1985; D. Rodrigues, 1985; Sommers & Collins, 1984; 
Womble, 1985). But researchers have only recently begun to investigate through 
controlled studies these claims for the benefiti. of using computers in writing classes. 

Researchers who have attempted to document the effects of computers on 
student writers have prouuced evidence that is at best inconclusive. Etchison (1985) 
found that computer classes made greater gains than control classes, with the 
computer students writing longer, better papers at the end of the term, even though 
they started out the term with scores well below the control group. Cohen (1986) 
reports that students who used word processing made 34% more revisions on end-* 
of-term essays than did the control students, even though the computer students 
were writing with pen and paper (which he interprets as an argument for transfer of 
effect from computers to pen-and-paper writing). 

In a program evaluation at Miami University of Ohio, Stonns (1986) reported 
no qualitative improvement in the essays of computer students as compared with 
control students; Storms does report a range of perceived benefits, including 
favorable teacher and student reactions, out-of-class computer use by students in 
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the experimental group, and more in-class writing in experimental sections. 
Hawisher (1987) reports no increase in revising activities for computer students; in 
fact, her pen and paper students actually revised somewhat more. She found no 
differences in quality between computer and paper-generated essays by her college 
students. Cross and Curey (1984) report inconsistent findings on measures of 
attitude, performance, process strategies, and grades across computer and regular 
groups. 

We are left with tepeated observations that both students and teachers 
appreciate using computers in their writing classes and believe such use results in 
improved writing anu .^vising, but the findings from controlled studies are at best 
equivocal. (For a more complete review of the research, see Bernhardt ^fia Wojahn, 
forthcoming.) Hav.isher (1986) reviews the research and concludes that the 
inconsistency is a result of research designs that are either not well concei /ed or not 
comparable with other studies, making it difficult to generalize. 

The present study is a program evaluation undertaken at Southern Illinois 
University-Carbondale (SIU-C) to assess the broad, measureable effects of using 
computers to teach introductory college composition. Twenty-four classes were 
studied— twelve control classes and twelve experimental— with the experimental 
computer classes meeting in the lab for half of their instructional time. Data on the 
success of the program were collected from a range of sources: pre- and posttests 
of student writing under both impromptu and take-home conditions; pre- and 
posttests of writing anxiety; records on attendance, tardiness, withdrawals, and 
homework and essay assignment completion; end-of-term course evaluation by both 
teachers and students; and self-report data collected from teacher meetings and 
teacher logs. (For discussion of writing program evaluation, see Davis, Scriven & 
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Thomas, 1981; and Wittc & Faiglcy, 1983.) 

METHODS 

Phviicil Stetting 

The study touk.place during Fall 1986, the second year after construction of 
two new IBM PC computer labs. The two adjacent labs each contained 32 PCs, 
with one lab set up as a classroom and the other as a drop-in lab. The computer 
classroom provided a PC for each student and software, including PC-Write (a word 
processing "shareware" program, which each student copied for persona! use), 
spelling and grammar check programs, and a range of other general applications 
software. The lab (and the classroom when not scheduled for classe'i) was open long 
hours: from 8 a.m. until midnight curing the first eight weeks Oi the term and from 
8 a.m. until 2 a.m. during the last eight weeks of the term, as student demand on 
the PC labs increased. Throughout the term students had access to PCs, though 
during popular hours in mid-afternoon and ea-' evening, there was often a wait of 
up to an hour. The lab and classroom were staffed by student workers and full- 
time staff hired by Computing Affairs. Whenever possible, the English Department 
assigned a graduate assistant to the classroom to help with classes. 

Twenty- four sections of aprroximately 22 students per section were studied: all 
were first semester, introductory composition classes in a two-semester sequence. 
We did not identify sections as computer classes in order to approach a random 
assignment of students to experimental and control groups. Twelve teachers each 
taught two sections, one computer and one regular. The time-table did not list 
teachers* names. 



Each of the computer classes met in the computer classroom one day per week 
and in a regular classroom one day per week, with each meeting lasting 75 minutes. 
All classes met on a Tuesday/Thursday schedule, and insofar as possible, teachers 
taught both sections in either the morning or afternoon. 

Iflstnictional Setting 

The computer and regular sections were in some ways the same course, having 
evolved within the same composition program. Instructors were generally expected 
to teach a process approach, with emphasis on revision through peer critiques and 
teacher commentary at various stages of writing. The recommended text was 
Scholes and Comley's The Practice of Writing. 2nd Edition: one teacher chose to use 
Moss' Composition bv Logic . Through the composition training program and 
because of the general pedagogical orientation of the department, the teachers were 
encouraged to take a rhetorical approach to writing, stressing the importance of 
purpose and audience by having students experiment with a wide variety of writing 
tasks through a general course sequence: from personal, reflective writing; through 
more objective, factual reporting; toward more critical, analytic, academic registers. 
The expectation was that students would do frequent, informal writing in addition 
to 6 or 7 longer, more formal texts. All teachers kept office hours and were 
encouraged to schedule individual conferences with students. The teachers often 
worked in the PC lab themselves and provided informal conferences to their 
students on demand. 

As a group, the teachers in the study were experienced writing teachers who 
received generally strong student evaluations, who displayed an interest in teaching 
with computers, and who were willing to be part of a study. Of the 12 teachers, 7 
had tauKht in the computer classroom during the previous year, these ? and an 



additional 4 others had attended a weekly seminar the previous fall on word 
processing and the teaching of writing; the twelfth teacher had taught composition 
with computers on another campus. Five of the teachers were lecturers; 7 were 
Experienced graduate student TAs« 

We did not urge teachers to attempt to teach the two courses in parallel 
fashion, with computers being the only variable. We recognized that the 
instructional setting would influence teaching strategirs: the nature and timing of 
assignments, the use of the textbooks* the use of small groups, the frequency of in- 
class writing, and so on. At a pre-t^rm meeting, the twelve teachers discussed what 
they anticipated might be different in a lab setting, with those who had taught in 
the lab offering advice to the others- We agreed to meet voluntarily throughout the 
term to discuss the sections, compare notes, solve problems, and share insights. For 
the first few weeks, we met every wcek^ and then every other week during the 
latter part of the term. 
Evalnativc Mgamrgs 

Pre- and posttests of student writing. Samples of students' impromptu and 
revised writing were collected during the second and the fourteenth week of the 
sixteen week course. For the impromptu sample, students were given one of two 
assigned tasks, told that the assignment would count as part of their course grade, 
and given forty-five minutes to write in class- When they were finished, the 
impromptu drafts were collected and photocopied by the researchers, who then 
returned the photocopies with instructions to take the drafts home over the weekend 
and revise them. Students were again told the revisions would count toward their 
course grades. Students had from their Thursday class meeting to the following 
Tuesday to revise their essays. 
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The posttest followed the same procedures. Computer students were given the 
impromptu task in the computer classroom and told they had the option of 
composing on disk. The drafts of ail the students were either copied onto disk or 
photocopied. Students were again asked on Thursday to rewrite, using what they 
had learned in the course to improve their essays, and to turn in their revised work 
on Tuesday. Computer students were told they could type or work w^th pen and 
paper if they chose. Revisions were collected on Tuesday, either on paper or disk. 

Two writing tasks were used in a split*halves design, with half the classes 
writing on Topic 1 for their pretests, and half writing on Topic 2. Topics were 
then reversed on the posttest. The two writing tasks had been pilot tested during 
the previous term, with a third topic being discarded because some students had 
difficulty responding to the task. The two remaining topics were designed to be 
comparable: they were similarly worded; both called for analysis and use of data 
presented graphically; both were two-part questions, asking students to generalize 
from the data presented and also to suggest what other data might be useful in 
supporting further generalizations about each topic (see the Appendix for the two 
topics). 

All full sets of two pretests and two posttests for each student were coded 
numerically for section, topic, student, and form. Complete sets included four 
samples from each of 146 computer students and 194 regular students. All essays 
were then keyed into disk files, with standard margins and spacing. Grammar, 
punctuation, paragraphing and spelling were left as written. Words or passages of 
text that had been deleted on the paper copy were not keyed. If students indicated 
insertions or transpositions, these moves were carried out. To the extent possible, 
the texts were copied to disk without enhancing them but reflecting the intentions 



of the authors. All essays were then printed for a uniform appearance. 

Seven readers were trained to rank sample essays, using papers from students in 
the study who had not completed all four tasks. A detailed scoring guide with 
criteria for each scale was developed and used throughout the scoring process. Two 
readers then independently scored each of the 1360 essays using a combined 
holistic/analytic scale with four variables scored on a 1-6 scale and two variables 
scored on a N3 scale: 

1 (low) to 6 (high) 

Holistic: overall impression of quality 

organizatioii: presence and effectiveness of discourse level planning, 

structure, signalling 
support development of generalizations with supporting detail, examples, 

evidence, reas< jed positions 
fluency: use of various and clear sentences, effective subordination, 
coordination, and transition 
1 (low) to 3 'high) 

conventions: use of standard English syntax, spelling, punctuation 
task degree of response to writing task as stated 
Where scores differed by more than a single point on a scale, a third reader scored 
the essay. The tie-breaking third score was used with the original score closest to 
it. Scores for each essay were averaged, producing a single score on each measure 
in a 1-6 range for the first four categories and a 1-3 range on conventions and task. 

Daly-Miller Writing Apprehension Test We administered the Daly-Miller 
V/riting Apprehension Test during the second week and again during the fourteenth 
week of chiss. 
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Records of Withdniwals, Attendance, Tardiness, and Assignment Co. pletion. 
Wc collected data on withdrawals from university registrar reports. A team 
researcher collected bi-weekly reports from each teacher on attendance, tardiness, 
and completion of major assignments. 

Stodcnt Evaluations. During the penultimate week of the course, students 
completed the short form of the ^'Instrument for Reporting Course and Teacher 
Effectiveness in College Writing Classes," a 21 -item questionnaire using a five-point 
scale to assess both course and teacher (Witte, 1981). Six additional questions were 
pilot tested during the previous spring term, revised, and then appended to the 
Witte instrument. They concerned use of textbook and handbook, getting to know 
classmates, writing grammatically correct papers, becoming a better writer, and 
enjoying writing. These additional questions were scored separately from the Witte 
instrument, which underwent factor analysis during its development. Direct 
questions about computers were a part of neither the Witte evaluation nor our added 
items. No reference wss made to computers. 

An additional questionnaire was developed for the comp^iter sections, with 
questions directly relevant to that experience. This questionnaire also used a five- 
point scale and was pilot tested during the previous spring term, with substantial 
revisions for wording and usefulness of the questions. A brief open-ended set of 
questions allowed for further student comments. All questionnaires wert 
administered by researchers who made it clear to students that teachers would 
receive no student feedback until after grades were filed. 

Teacher Evaluations. On going records were compiled from notes from the bi- 
weekly teacher meetings and on an ad hoc basis as teachers commented to the 
researchers on their courses. Teachers were encouraged to keep logs throughout the 
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tenn; four did so and these became a part of the data. At the end of the term, 
teachers completed a comparative evaluation of their two courses: 33 questions that 
had been developed through pilot testing the previous fall and spring. Additional 
written comments were also solicited. 

Class Observations. Both classes of each of two teachers were observed 
continuously throughout the term, with a detailed t*meline of minute-by-minute 
activitiei and an accompanying log of notes kept for each of the four sections. 
These data will be presented elsewhere as an observational study but are mentioned 
here because they did influence our sense of the program and discussions during 
teacher meetings. 

Data Collection. Students in both regular and computer sections were told 
dunng the first week of class that writing samples, course evaluations, and measures 
of their attitudes about writing would be collected because the English Department 
was. interested in lea^-ning how our programs were working and how they might be 
improved. It was stressed that our intention was not to evaluate individual students 
and that any data collected would be anonymous. 

Research assistants, not the classroom teachers, administered all pre- and 
posttests, questionnaires, and end-of-term evaluations to ensure that all classes 
received the same instructions and were tested under the same conditions. 
Directions for testing and assessment were printed and read by the research 
assistants; they offered standard answers to any q^^estions. 
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RESULTS 
Pre- and portterta of rtadent writing 

Interrater reliabilities were calculated for ihe four 6-point scales using Pearson's 

r 

HoV-^^c .74 
Organization .72 
Support .71 
Fluency .66 

With the two 3-point scales, conventions and task, percentage of exact agreement 
was calculated: 

Conventions 63% 

Task 96% 

The high percentage of exact agreement on task was the result of sharply defined 
criteria. The two-part task entailed generalizing from a data set and suggesting 
what other data would be necessary to form a complete position. Students either 
responded to both parts of the task (3), to one but not the other (2), or did neither 
but responded in some way not called for by the task (1). 

Table 1 presents regular and computer posttest means for Impromptu, Revision, 
and Improvement Scores (Improvement « Revision - Impromptu). The posttest 
scores were adjusted for pretest scores in order to eliminate the difference between 
the groups in their initial abilities. This allows us to simulate an experiment where 
treatments are applied to groups od equal abilities. 

[Insert Table 1 about here] 
On most measures, students in the computer sections performed slightly worse than 
the regular students on the impromptu portion of the posttest. But the computer 
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students then improved their scores on the take*home revision so that they ended up 
with revision scores somewhat higher than the regular students. In no case, 
however, is the difference between the groups on either impromptu or revision 
scores statistically significant. 

The improvement score represents the differenpe between impromptu and 
revision, our measure of ability to improve a first draft through take-home revision. 
For the macro-level discourse features— holistic score, organization, and support— 
the computer students improved their essays to a greater extent than did the regular 
students. The two groups we e comparable on their improvement of fluency and 
conventions; the computer group improved the task-responsiveness of their essays to 
bring them to almost the same level as the regular students. 

Table 2 presents the results of six separate analyses of covariance, where 
variance in improvement scores for each of the six scaled variables served as the 
dependent variable. Three independent variables were defined in the nodel in 
addition to the error term: method (computer vs. regular), instructor, and interaction 
of instructor with method. The covariate pretest score was included in the model to 
reduce variability of the experimental error. The degrees of freedom for all error 
terms was 305. 

[Insert Table 2 about here] 
The instructor is the strongest effoct, followed by the interaction of instructor with 
method, with four of the six scales showing significant contributions at the .005 
level (df«Il). Even after accounting for the contributions of teacher and the 
interaction of teacher with method, method alone still contributes significantly to 
variance in posttest holistic improvement scores (p < .05; df»l), and in task- 
responsiveness (p < .05; df while the F values for organization and support 
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approach significance. Improvement in fluency and conventions do not appear to be 
much affected by any of the factors in the model. 

Note that the improvement scores on the pretest (students' abiliti3s at th 
beginning of the term to revise and impro^'s holistic scores, organization, etc.) do 
not appear to contribute significantly to the variance on posttest improvement 
scores, with the exceptions of support and task scales. Table 3 presents Pearson 
correlations for pretest improvement and posttest improvement scores. 

[Insert Table 3 about here] 
These low correlations suggest that ability to improve essays through revising at the 
beginning of the term is not highly correlated with ability to do so at the end of 
term. In other words, learning appears to take place. 

When the computer students wrote their posttest essays, they bad the option of 
using eitiier pen and paper or the computer to draft and revise their work. Table 4 
presents t-tests of differences in improvement scores for paper revisers and 
computer revisers within the experimental treatment. 

[Insert Table 4 about here] 
The computer revisers improved their impromptu drrfts during revision more than 
the paper revisers in every category except task. Those students who chose to revise 
on the computer at the end of the term significantly improved (at the .OS level of 
confidence) the overall holistic quality, the organization, and the level of support as 
compared to those students who chose to revise on paper. The improvement scores 
of the computer revisers were larger, too, for fluency and conventions, though these 
measures did not achieve significance. The computer revisers' improvement scores 
we«*e in every case larger than the average scores for the whole group of students in 
the ' tudy. 
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Driv-Milkr Writing Apprcbcmion Tes^ 

Results for the Daiy-Milier Writing Apprehension Test are presented in Table 5. 
We used a response scale which reverses the normal orientation, so higher scores 
mean higher appishension. 

[Insert lable 5 about here] 
Controlling for prescore differences and comparing least squares means yields an F 
value of 1.11 (p > .29; df « 1), a finding of no significant difference between 
computer and regular students on post anxiety scores. Both groups moved in the 
same direction at about the same magnitude, toward slightly increased anxiety. 

Records of Withdrawals, Attendance. Tardiness, and Alignment Completion 

Table 6 presents figures that compare regular and computer sections on 
withdrawals, attendance, tardiness, and assignment completion. Not all teachers 
kept complete records so there is some missi data. 

[Insert Table 6 about here] 
In all cases, the data on the computer sections are somewhat worse than those for 
the regular sections. Though mcrt regular students withdrew during the first week, 
the number of computer students withdrawing was higher during weeks 2-5 and 6- 
9, for a larger overall percentage. Attendance, too, was worse in computer sections, 
as was tardiness. Finally, the percentage of computer students not completing major 
writing assignments was double that of the regular students. Regular students also 
completed more of their homework assignments than did the computer students. 

Student Evaluations 

The short form of the "Instrument for Reporting Course and Teacher 
Effectiveness in College Writing Classes'* loads all 21 items on two broad factors- 
teaching and course effectiveness. Table 7 presents the means for regular and 
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computer groups. 

[Insert Table 7 about here] 
On average, the computer students rated their instructors slightly higher than did 
the regular students; the regular students responded slightly more favorably to 
questions concerning content. Neither difference was significant. 

Evsluatioii of Teaching. For the twelve matched sections, six instructors were 
favored by their computer sections, six by their regular sections. Ten of 18 
statements concerning the instructor were rated more positively by students in 
computer sections. The student responses most strongly favoring their computer 
teachers included items describing the teacher as 1) helpful, 2) good at using class 
time to help them as they wrote, 3) intellectually stimulating, 4) good at teaching 
how to support ideas with examples and details, 5) good at writing comments on 
papers that were easy to understand, and 6) r fair evaluator. 

Students in regular sectior^s responded more favorably to five of the 18 
statements concerning their instructors. The student responses most strongly 
favoring their regular teachers included items describing the teacher as 1) good at 
trying to 'ncrease their confidence about writing, 2) good at teaching them to 
consider audience (responses were strong for both methods), and 3) good at teaching 
students to write different kinds of papers. 

Evaluation of Course Content A large majority of studencs in both computer 
and regular sections agreed that the course was currently useful to them, that what 
they learned in the course was valuable, and that the course would be useful to 
them in the future. 

Students in regular sections appreciated their textbook and their grammar 
handbook more than students in computer sections, while more computer stude/its 
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agreed that they learned to write grammatically correct papers. 

Students in computer and regular sections responded similarly to the statement 
that they got to know and work with their classmates. Comparable numbers also 
agreed that they were better writers at the end of the course than they were at the 
beginning. To the statement that they enjoyed writing more at the end of the 
semester, responses were almost evenly divided for both computer and regular 
sections. 

Responses of Computer Students to the Use of Computers. An additional 
questionnaire asked computer students to evaluate the usefulness of computers for 
writing. Overwhelmingly, the students thought computers were a good writing tool. 
About hdf agreed that they preferred composing at the computer, with a third 
reporting that they did noi usually write pen and paper drafts prior to using the 
computer. Two-thirds preferred to revise on the computer rather than with pen and 
paper. By the end of the semester, 78% agreed that they felt comfortable working 
with the word processor. 

Responses showed that most students were not new to computers, with 43% 
saying they had used word processing prior to the course. Most students took 
advantage of their word processing skills for other courses and a third reported 
teaching others to use PC- Write. 

Supporting software was available to all students, yet the majority of students 
regularly chose to rely on the word processor alone. Not quite half used the spell 
checker often. Far fewer took advantage of the grammar software. 

Many students (52%) feit that they spent more time on the course than their 
friends in regular sections. But about half also believed that they made a better 
grade because of the word processor. 
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Although some people fear that the computer may de-humanize the writing 
classroom, the students in computer sections reported otherwise. Only 11% agreed 
that having computers in the classroom created a barrier between teachers and 
students. Only 16% felt that having computers in the classroom placed too much 
emphasis on machines and not enough on people. 

Despite the distractions inherent in computer classroom settings, many students 
found the microlab a comfortable place to write; a third disagreed. Nevertheless, 
two-thirds agreed that they liked having their class meet in the microlab. 

Respoiise$ of Students to Open Ended Questions. Students' responses to open- 
ended questions were overwhelmingly positive. They liked the ease of revising, 
editing, and proofreading; the time they saved; tne neatness of their papers at all 
stages; and the increased freedom in organizing. Most stated that using the 
computer caused them to change the way the planned, organized, wrote, revised, 
and edited their papers. They said that they now took more time to seriously 
consider their writing and more time to take their papers through various drafts. 
Many mentioned that spending extra time was worth it. 

Access to the computers and lack of familiarity with function keys were the 
two problems or frustrations mentioned most frequently. Students overwhelmingly 
s.wted that lack of access to computers was the worst thing about writing with the 
computer. The second major problem was losing files. Students also complained 
that papers look different on screen than they do on paper, ihat it takes those who 
don't know how to type extra time to write papers, and that the lab is not the ideal 
place to concentrate on ideas. 

The most typical advice students offered to their teachers or lab assistants was 
that they should better help students learn how to use the various commands and the 
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available software. Other advice was to use the microlab solely as a workshop 
environment, with no lecture time. Students also requested that not all minor 
assignments be done on computer since access to computers was a problem. 

Most felt the English Department should continue using computers, with some 
stating that all writing classes should meet in the lab. Students recognized the 
importance of computers in their futures: 83% said that would continue to use word 
processing. 

Teacher Evaluations 

Responses of the twelve teachers to an end-of-term questionnaire are ranked in 
Table 7, beginning with items reflecting highest agreement and moving to lowest. 
To simplify presentation, "strongly agree" and "a^^ree" are collapsed under "agree" 
and similarly for "disagree." Possible means range from 5 (strongly agree) to 1 
(strongly disagree). 

[Insert Table 8 about here] 
Most teachers indicated they preferred to teach in the computer classroom and all 
would like to do so again, feeling comfortable and in control of their classes. Most 
noted that student attitude was better and that the c Dmputer classroom was more 
conducive for helping students with their writing while it was still in progress. 
Teachers felt closer to their students and appreciated the workshop setting. The 
teachers felt strongly that a lab assistant was necessary and that more machine access 
would help. 

In the computer sections, teachers required more in-class writing, slightly more 
prewriting, and more daily assignments. Teachers tended to agree that studenta 
were doing more revising, even though they were close to neutral on the question of 
whether they were requiring more revising. In the teachers' judgement, the students 
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were comfortable using the word processor and actually did so for most assignments. 

DISCUSSION 

« 
« 

In general, our results favor the use of computers in teaching composition. The 
pre* and posttest comparisons of impromptu writing show that the computer 
students did not perform quite as well as the regular students at the end of the term 
when asked to write an impromptu essay. But after revising, the scores of the 
computer group were higher than those of the regular group. Tht computer group 
revised their work to make significantly greater improvements during revision when 
compared to the control students. While significantly greater improvement was 
found for holistic scores, the computer students also improved their essays during 
revision (at levels approaching significance) on the high-level discourse features of 
organization and support. Significantly better task responsiveness was also 
characteristic of the revising of the computer students— during revision, they were 
more likely to bring their essays into accord with the assigned task. Very little 
effect was apparent for low level features— sentence fluency and conventions. 
These findings suggest that the computer, considered by itself, does have a positive 
effect on student revising skills, especially discourse level skills. 

One reason the computer students were able to improve their essays more than 
the regular students was because their impromptus were generally poorer than those 
of the regTilar students. Perhaps becoming familiar with the machine's usefulness as 
a revising tool encouraged students to write quick, rough drafts, which they knew 
they could revise later. Whatever the explanation, the computer did help the 
students revise their work to the point where it was a little better than that of the 
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regular students. 

As a natural consequence of the experimental design, the computer students 
self -selected into two groups at the end of the term, those who chose to do their 
revising on screen and those who chose to revise on paper. Striking differences 
characterize thejse two groups* abilities to improve their essays on revision, with the 
computer revisers significantly outperforming the paper revisers on every measure 
except task responsiveness. All improvement scores for the computer revisers-* 
including the holistic evaluation and the five analytic scales-*were well above the 
average for all students in the study. 

These fmdings suggest differences in adaptation to the technology, with some 
students (112 of 146, or 77%) becoming comfortable with the computer and finding 
ways to make it work as a revising tool. The important gains for this successful 
group would be obscured were their scores simply averaged with those of the 
smaller group who chose not to revise with the computer on the posttests. These 
findings echo Herrmann's findings with high school students (1987); she identified 
three groups who adapted with varying success to writing on the computer. There 
does not seem to be a simple relation of machine to improvement; instead one group 
takes ownership of the computer and uses it to good purpose, while a second group 
does not. 

Though the data favored the use of computers considered in isolation, the 
covariate analysis showed stronger effects for teacher and for the interaction of the 
teacher with the method of using computers. The teacher had a very strong effect 
on whether the students improved. If our goal were simply improved student 
writing, we would probably get better results from choosing talented teachers or 
from training our teachers than from introducing computers into the classroom. If 
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we do introduce computers, we need to work with ^e teachers to encourage 
adaptation to the technology. As other reseaichers have noted, we cannot simply 
put computers in a room and expect to see dramatic improvement in student writing 
and revising. Teachers, not machines, have the strongest effects on student writing 
improvement. 

The attitudinal data suggest that both teachers and students viewed their 
experiences in the com.<>uter classroom positively. Few considered the computer a 
barrier between teachers and students, and few found the computer a hindrance to 
learning. Students seemed to appreciate learning both writing and word processing, 
skills they immediately applied both in and outside the composition classroom. 

The teachers received slightly higher ratings from their computer sections than 
from their regular sections. Thir .lay reflect the teaching situation in a lab, where 
students receive more individual attention as they write— teachers can move about 
more freely, reading and discussing the student's writing as it is displayed on the 
screen. More positive student responses to instructors of computer sections may 
reflect the more intimate classroom interaction, with students and instructors 
working together more directly, allowing them to know and understand one another. 
Though the teachers felt the computer classes took longer to come together as 
learning groups, in the end, many instructors felt closer to their computer students. 
Teachers were certainly physically closer as they helped students write, sometimes 
offering a pat on the back, sometimes kneeling to make eye contact. 

Teachers noted that students in their computer sections seemed to have better 
attitudes. Most believed that this was due to the ease of writing with a computers- 
students were able to locate problems in their writing and make changes quickly and 
neatly. Instructors also found students in computer sections more willing to revise 
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and believed that the computer students did more prewriting and revtsing--even 
when it was not formally required. In spite of some negative feelinjs* -about 
limited access to machines, about having to spend more time working on their 
essays, and about learning to use word processing (and invariably losing files)— -the 
majority stated that they liked meeting in the computer classroom and that they 
liked using the computer to write. 

There are certainly drawbacks to meeting in a lab environment. The students 
spend more time writing, but receive fewer lectures. Less lecture time in their 
computer classes caused instructors to feel rushed; some felt that the students in 
computer sections missed important material. However, the need to focus and 
organize their lecture or discussion time might actually be viewed positively*-many 
teachers reconsidered and streamlined their practiced lectures and usual assignments. 
And the quality of the students' writing did not appear to suffer from having less 
lecture time. 

Further drawbacks are indicated by the consistently poorer showing of the 
computer students on measures of course withdrawal, attendance, tardiness, and 
assignment completion. From our pilot studies and from the reports of other 
researchers, we know that studci-tts enjoy meeting in the lab. We had often 
observed students arriving early and staying late, with some even staying through 
two full classes. So we were not prepared for data which showed that student 
attendance and tardiness were worse for the computer CioSScS. 

The instructors felt that the workshop atmosphere of the computer classroom 
affected student attitudes toward coming on time, or indeed, toward coming at all. 
While our teachers readily endorsed the value of a workshop classroom that focused 
on work in progress, students seemed slower to perceive the value. Many students 
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preferred to write on their own time and viewed a workshop as "nothing being 
taught." Students also found they could easily enter the lab late and sit down at a 
computer, with little or no disturbance to others in the room. 

Among those students who were in attendance, teachers repeatedly noted 
reluctance to engage in discussion or structured activities. In the lab, students 
expected to work on their texts with as little interference as possible. They did not 
want to discuss writing, do exercises, or answer questions on their reading. The 
coinputers exerted a strong draw on student attention, making it difficult for 
teachers to do anything except allow students to work individually. 

The instructors noted in their logs and in weekly round-table discussions that 
their students often lost patience because of the long waiting perird to gain access 
to computers. Some assignments may not have been completed because of limited 
time on the computers. Teachers and students agreed that free access to machines is 
important. 

Some of the results of our study were surprising. We expected students in 
computer sections to see their class as more valuable and more useful to their 
futures, yet the responses to the evaluation questionnaire did not confirm this 
expectation. Likewise, we expected students in the computer sections to rank their 
classes higher for its interactive aspect— working with other students— but such was 
not the perception. Again, students simpjy may not share our teacherly perceptions 
of the value of a collaborative class. 

These results on student attitudes differ from our pilot study results, which 
indicated that students overwhelmingly preferred having their composition classes 
meet in a computer classroom. Pilot study students in computer sections had much 
more positive attitudes toward writing, toward their course and toward their 
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teachers, than did the students in regular sections, giving both course and teacher 
more favorable ratings on every item but one. (The regular students in the pilot 
study found their handbook and textba.^k to be more useful than did the computer 
students.) 

We believe the differences between the pilot study and the present study can 
largely bt accounted for by the control of assigning each teacher to matched 
sections. Student evaluations appear overwhelmingly dependent on the teacher—the 
same teacher, even when methods are very different, receives approximately the 
same evaluation. This might suggest that we should view the glowing reports from 
other studies with caution-*the more positive attitudes of students using computers 
in composition classrooms might largely be due to the teachers themselves, even 
when teachers in control sections are teaching from the same syllabus, using the 
same teaching approach, and so on. We might also assume a halo effect, with the 
initial enthusiasm among computer students and teachers giving way to more neutral 
attitudes after several terms. 

While we think that matching teachers with sections successfully controlled for 
teacher effect, this control also exerted strong and unintended effects on the whole 
experience of computer classrooms. For the most part, instructors struggled to keep 
their computer sections parallel with their regular sections. Given a choice, most 
busy teacher/scholars prefer teaching one preparation twice, rather than preparing 
for two different classes. If teaches s had taught two computer sections, we believe 
we would have seen even more changes in strategies, assignments, and course 
requirements, and, we imagine, stronger effects on writing and attitudes. Instead, 
teachers worked to keep the students doing the same work on the same schedule. 
We saw a real change from the pilot study, when computer teachers talked about 
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how they needed to reduce the number of assignments, change their strategies to fit 
the microlab environment, and design assignments to take advantage of the 
collaborative possibilities of the computer classroom. During the study itself, we 
watched teachers do everything they could to make the classes the same, in spite of 
our urgings to adapt as necessary. 

This study provides important support for using computers to teach college 
composition, and future studies should incorporate several features of this study. 
Students shc^ 1 be tested for revision skills as weli as impromptu skills, since this is 
where we should expect to see the most benefits of the technology. Students also 
should be allowed to compose on computers for evaluation. While we should 
continue to recognize the importance of s .titudes, we should also continue to collect 
data from writing evaluation and unobtrusive measurei ^{ course success (such as 
attendance or assignment completion). 

Future studies should test systematically for the unintended results from this 
study. Specific studies should be designed to distinguish subgroups among students 
using computers to tease out the differences between those who adapt well to the 
technology and those who do not. Likewise, we should study and attempt to isolate 
what it is Chat determines how well individual teachers adapt to a lab environment, 
rer -'nizing that we should not expect all teachers to be comfortable and successful 
in a lab setting. We should study how teachers change their strategies when they 
are free to adapt instruction to a lab setting by removing the constraint of double 
preparation. This suggests open, naturalistic investigation of teachers in lab settings 
to hypothesize and define variation and adaptation. 

Finally, we need tu acknowledge that we may not be able to measure 
confidently the effects of such a powerful writing tool on such a complex skill as 
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writing ability. The literature on testing for student learning over the space of a 
term*-whatever the experimental treatment and whatever skill is considered the 
dependent variable—is equivocal at best. The effects of computers on writing 
ability may not be a matter of quick transfer, but of subtle and incremental 
evolution over the life of a writer (See Perkins; 1985). The real results of 
introducing student writers to computers may be realized over the long term, as 
students continue to grow as writers and become increasingly proficient at using the 
machine to enhance their writing processes and products. 
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Appendix 
Topics for "Pre^ and Basttests 

Topic #1 



TIT]? ifrrM ftlygtfiac and Book of Facts 1985 (p. 113) presented the follodng 
infonation about various U. S. cities: 



Quality of Life in U.S. Metrqpolitan Areas: A Qonparative Table* 
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Atlanta, Ga. 


U,590 


4.3 


3.1 


3.5 


108-149 


58 


86.4 


Buffalo, N.Y. 
Ctiiaqo, H. 


11,160 


8.5 




1.0 


55-208 


132 


77.6 


13,069 


8.7 


0.6 


1.6 


86-173 


99 


82.3 


Dallas-Ft.Wbrth, Tx. 


13,846 


3.9 


2.7 


3.6 


138-132 


41 


96.1 


Denver-Boulder, Co. 


13,964 


4.1 


2.5 


3.2 


llV-120 


157 


85.8 


Honolulu, Ha. 
Milwaukee, wis. 


12,130 


4.7 


1.8 


2.4 


86-100 


0 


87.4 


12,597 


6.6 


0.7 


1.6 


94-172 


144 


79.7 


Rioenix, Ariz. 


11,086 


3.9 


2.9 


3.6 


213-071 


10 


102.2 


Salt late City- 
















Od^enUt. 


9,670 


6.1 


1.7 


2.6 


127-135 


128 


90.2 


San Francisco, Ca. 


17,131 


6.2 


1.4 


2.3 


162-104 


0 


68.2 


Seatae-Everett, Wa. 


13.239 


8.4 


1.5 


3.0 


71-201 


16 


74.0 



Ohe chart ubove suninarizes information that could be used to determine 
which cities are the best places to live. Iteing the information given, 
difldifls in an essay how these factors (such as per capita perscrol 
inxne) night relate to the qiality of life. Also include suggesti<^ 
concerning \ihat additional factors are important in conparing the guedity 
of various cities. 
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Topic «2 



USA Ttadav (May 5, 1986) asked American families about their vacation 
plans. Ihey responSed as follows: 



How mich will you sp&nd? 
500 or less 



501-1, 000.^ 

1 , 001-1 , 50 0 11% 



2^ 



1,501-2,500_ 
2,501-ar moac e 
Don^t know or 

wouldn^t sa y 7% 



JL2I 



Hew will you travel?* / 
airplane 



our car_ 
rented car 



23% 



tc^dn or bus 10% 
canper or PV gi 

^nore than one answer in some 



Mhat will you do? 

Sightseeing 

rest, relate 28% 

swim, water qports 

sunbathe, beach 19% 

visit friends/ 
relatives 

socializ e 14% 



J2i 



play sport s 10% 
fishing, huntin g 10% 
canpincr/hiJdn a 9% 



Nhere will you go? 
Florida^ 



California. 
Texas. 
BahanBS 
Europe. 
Canada 
Hawaii 



New York 



MichicE m 3% 



10% 



Ihe chart above sumsarizes information about family vacations. Using the 
information given, discuss in an essay what seese to be the typical 
family vacation. Also include suggestions concerning \ihat additional 
information is necessary to give a conplete description of a typical 
family vacatiai. 
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Table 1 
Mjusted PoGttest Means 
Regulea: vs. Ocnputer 

Inprcnptu Revisicn Inprovaoaent 

Holistic Regular 3.29 3.70 .41 

Oonputer 3.17 3.78 .61 

Organization Reg 3.31 3.65 .34 

Ocnp 3.19 3.68 .49 

Sufport Reg 3.31 3.70 .39 

CCnp 3.26 3.84 .58 

Fluency Reg 3.27 3.70 .43 

COB^ 3.29 3.71 .42 

conventions Reg 2.24 2.35 .11 

Ocnp 2.26 2.38 .12 

Task Reg 2.4 2.50 .03 

Ocnp 2.37 2.52 .15 
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Table 2 

F Valites for Analysis of Varianse 
Using IntJEOveaoient trm Pcssttest Imar t aayto i to 
Posttest Revision with Rretest Scores Oontzolled 

Method Instructor/ Instructor Pretest 
{DP=1) Method (DF^ll) (DP=1) 

(DP=11) 



Holistic 



4.59* 2.64** 3.71** 1.15 



Organization 2.38 2.86** 4.10** 



.16 



Support 



3.64 2.71** 3.79** 6.81** 



Fluency 



.00 1.15 



1.11 



.79 



Oonventions .02 1.35 



1.78 



.27 



Task 



5.23* 1.92* 3.49** 4.23* 



* p < .05 
** p < .005 



Table 3 

Ftearson Oorxelations for Qpprwanent Scxyres 
on Pre- and Posttests 



n = 330 

Holistic .07 

Organization .02 

Si^port .12 

Fluency .06 

C3onventic3ns .01 

Task .04 
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Table 4 
T-Tests for BetMeen-Gnxp 
Differences in Liprovenent Scores: 
Vaper vs. (Xnputer Revisers 

All Paper OGD(uter -t 

Subjects Revisers Revisers 
(n=330) (n=34) (n=112) 



Holistic 



.48 



.69 -2.38* 



Organizaticn .41 .73 .58 -2.22* 



Support 



.47 .22 



.66 -2.31* 



Fluency 



.42 .26 



.45 -.98 



Oonventiois .12 



.00 



.16 -1.36 



Task 



.09 .24 



.12 1.18 



* p < .05 

Assuming unegiial variances 
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Table 5 
Test of Writing Apprahetsion 



(n) Pre Post 



OoniMter Students 182 84.2 86.6 

Regular Students 207 78.8 81.3 

Note: Higher scx>re = higher apprdiensicn 
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Table 6 

WithdraMaOs, Attendanoe, Tardiness, and Assignment OGnpletion 

WitfadnMls 

(12 regular; 12 cxxifxiter classes) 

Regular Ocwputer 



Total 33 (11.5%) 40 (14%) 

First week 17 ii 

Weelcs 2-5 5 12 

Vteeks 6-9 11 17 



Attendanoe 

(10 regular; 10 ocqputer classes) 
Average % absent 9.02 9.51 

First half term 8.15 8.62 

Ser ^od half tern 9.89 10.40 



TEUCdiness 

(10 regular; 10 ocnpoter classes) 
Average % tardy 3.12 4.39 

First half term 2.36 3.78 

Secxnd half term 3.88 5.00 



39 



Ncn-Oonpleticn of Nkriting AasiignKnts 

Percentage of stutoits not tumixrr in work: 

Ragular CXjc|xiter 

Major Assignments 4.18 8.45 

(11 regular; 11 coiputer classes) 

}iam/K>rk assignonents 9.87 14.62 
(7 regular; 7 oonputer classes) 



l^le 7 
Results of Ocurse Evaluation 



Teacher ef fectivaiess 



Oaapatet 



2.52 



Regular 



2.55 



Oourse eff Ar» iw>nogg 



Mean 



Ocnputer 



2.21 



Regular 



2.17 



Note: Biese are grouped means for all itaaas relating to teacher 
and all itans relating to oourse effectiveness, calculated from 
a 1 to 5 scale, v^iere 1 is strongly agree and 5 is strongly 
disagree. Polarity has been reversed on negatively worded items. 
Ixwer means indicate more positive evaluation. 



41 



p. 39 



Table 8 

Reqxxises of Teachers to the Use of Ctnpiters 

Mean Agree Neutral Disagree 



1. 


Students had probleoas getting 


4.67 


12 


0 


0 




to a nachine: 










2. 


I'd lUoe to teach in the lab again: 


4.58 


10 


2 


0 


3. 


I waUced around the rcxn more: 


4.58 


11 


1 


0 


4. 


Students did all their assignments 


4.42 


5 


7 


0 




cn the miciTCKxntJUter; 










5. 


I had to streamline and focus m/ 


4.42 


11 


1 


0 




lectures/discussions 










6. 


Hhe setting edloMed for more interaction 


4.25 


11 


0 


1 




between me and the students 










7. 


The setting allowed for more 


4.25 


10 


2 


. 0 




interaction among students 










8. 


I required more in-dass writing: 


4.08 


11 


0 


1 


9. 


I made specific assignments that 


4.08 


9 


3 


0 




required the use of the microocnputer: 










10. 


A lab assistant is necessary: 


4.08 


8 


2 


2 


11. 


I enjoyed teaching this section more: 


4.00 


9 


0 


3 


12. 


I lectured less: 


4.00 


8 


3 


1 


13. 


Students revised more: 


3.92 


9 


3 


0 


14. 


Students were more confident writing 


3.83 


7 


5 


0 




cr. U : ainputer as oijposed to pen and 












paper iTf the end of the oourse: 










15. 


I chectod tha student writing more: 


3.75 


8 


3 


1 


16. 


Students did scare prewriting: 


3.75 


8 


3 


1 
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17. Students needed more InstnxAion on 
the machine: 

18. Stutott attitude was better: 

19. I touched the stutJents more: 

20. I was stricter abcwt the appearance 
of the pdpers'. 

21. I felt more in control of the class: 

22. I felt closer to these students: 

23. Grading was easier: 

24. Stuients received less information 
about writing: 

25. I required more presnriting: 

26. I assigned more daily writing 

27. Students felt more arodcus: 
28 I felt more comfortable: 

29. I created aissignments on disk and 
transferred than to student disks: 

30. I required more revising: 

31. Stixtent ideas were better: 

32. Students tamed in fewer late papers: 

33. I required fewer major thanes: 
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3.42 
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3.42 


7 
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3.42 
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3 


3.33 
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2 


3 



3.25 


5 


4 


3 


3.17 


5 


4 


3 


3.17 


4 


6 


2 


3.08 


3 


5 


4 


3.08 


6 


0 


6 


2.92 


4 


4 


4 


2.75 


0 


10 


2 


2.33 


1 


6 


5 


2.00 


2 


1 


9 
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